Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use wasi crate for Core API #63676

Merged
merged 23 commits into from
Sep 6, 2019
Merged

Use wasi crate for Core API #63676

merged 23 commits into from
Sep 6, 2019

Conversation

newpavlov
Copy link
Contributor

@newpavlov newpavlov commented Aug 18, 2019

@rust-highfive
Copy link
Collaborator

r? @Kimundi

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 18, 2019
@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-18T03:45:23.2093028Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-18T03:45:23.2276693Z ##[command]git config gc.auto 0
2019-08-18T03:45:23.2359758Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-18T03:45:23.2432225Z ##[command]git config --get-all http.proxy
2019-08-18T03:45:23.2581654Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63676/merge:refs/remotes/pull/63676/merge
---
2019-08-18T03:45:58.1236850Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-18T03:45:58.1238276Z 
2019-08-18T03:45:58.1239916Z   git checkout -b <new-branch-name>
2019-08-18T03:45:58.1241259Z 
2019-08-18T03:45:58.1242655Z HEAD is now at 98e920dc7 Merge 6c299244305cdf585e515d608cc2bafa31081475 into bd1da18b04afba5dfc09ad1b56df3285f1d039c3
2019-08-18T03:45:58.1403710Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-18T03:45:58.1407137Z ==============================================================================
2019-08-18T03:45:58.1407202Z Task         : Bash
2019-08-18T03:45:58.1407270Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-18T03:51:54.6947258Z     Checking rustc-std-workspace-alloc v1.0.0 (/checkout/src/tools/rustc-std-workspace-alloc)
2019-08-18T03:51:54.9078611Z     Checking panic_unwind v0.0.0 (/checkout/src/libpanic_unwind)
2019-08-18T03:51:55.0511394Z     Checking hashbrown v0.5.0
2019-08-18T03:51:57.5337715Z     Checking backtrace v0.3.34
2019-08-18T03:51:58.1446193Z error: unexpected close delimiter: `)`
2019-08-18T03:51:58.1446595Z    --> src/libstd/sys/wasi/fd.rs:232:44
2019-08-18T03:51:58.1446901Z     |
2019-08-18T03:51:58.1447247Z 232 |         wasi::sock_shutdown(self.fd, how) }).map_err(From::from)
2019-08-18T03:51:58.1448057Z 
2019-08-18T03:51:58.1448387Z error: incorrect close delimiter: `)`
2019-08-18T03:51:58.1448387Z error: incorrect close delimiter: `)`
2019-08-18T03:51:58.1448679Z    --> src/libstd/sys/wasi/fd.rs:141:10
2019-08-18T03:51:58.1448947Z     |
2019-08-18T03:51:58.1449253Z 132 |     ) -> io::Result<WasiFd> {
2019-08-18T03:51:58.1449630Z     |                             - un-closed delimiter
2019-08-18T03:51:58.1450161Z 141 |         ))?;
2019-08-18T03:51:58.1450531Z     |          ^ incorrect close delimiter
2019-08-18T03:51:58.1450575Z 
2019-08-18T03:51:58.1685847Z error: aborting due to 2 previous errors
2019-08-18T03:51:58.1685847Z error: aborting due to 2 previous errors
2019-08-18T03:51:58.1685948Z 
2019-08-18T03:51:58.1776134Z error: Could not compile `std`.
2019-08-18T03:51:58.1776486Z 
2019-08-18T03:51:58.1776964Z To learn more, run the command again with --verbose.
2019-08-18T03:51:58.1833205Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-08-18T03:51:58.1835065Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-08-18T03:51:58.1835216Z Build completed unsuccessfully in 0:03:07
2019-08-18T03:51:58.1889349Z == clock drift check ==
2019-08-18T03:51:58.1906283Z   local time: Sun Aug 18 03:51:58 UTC 2019
2019-08-18T03:51:58.1906283Z   local time: Sun Aug 18 03:51:58 UTC 2019
2019-08-18T03:51:58.3399732Z   network time: Sun, 18 Aug 2019 03:51:58 GMT
2019-08-18T03:51:58.3401192Z == end clock drift check ==
2019-08-18T03:52:01.5379249Z ##[error]Bash exited with code '1'.
2019-08-18T03:52:01.5412911Z ##[section]Starting: Checkout
2019-08-18T03:52:01.5414349Z ==============================================================================
2019-08-18T03:52:01.5414413Z Task         : Get sources
2019-08-18T03:52:01.5414454Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-18T06:29:21.9637757Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-18T06:29:21.9797868Z ##[command]git config gc.auto 0
2019-08-18T06:29:21.9866701Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-18T06:29:21.9918446Z ##[command]git config --get-all http.proxy
2019-08-18T06:29:22.0057088Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63676/merge:refs/remotes/pull/63676/merge
---
2019-08-18T06:29:55.8442425Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-18T06:29:55.8442455Z 
2019-08-18T06:29:55.8442656Z   git checkout -b <new-branch-name>
2019-08-18T06:29:55.8442684Z 
2019-08-18T06:29:55.8442751Z HEAD is now at a4b25866b Merge 7bfb3d174bae45db93aae1cbdbfe6ce46a22b068 into fc8765d6d8623b2b5b4ca1d526ed1d7beb3fce18
2019-08-18T06:29:55.8603083Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-18T06:29:55.8605699Z ==============================================================================
2019-08-18T06:29:55.8605752Z Task         : Bash
2019-08-18T06:29:55.8605813Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-18T06:35:29.2459866Z     Checking backtrace v0.3.34
2019-08-18T06:35:30.9420821Z     Checking rustc-std-workspace-alloc v1.0.0 (/checkout/src/tools/rustc-std-workspace-alloc)
2019-08-18T06:35:31.0247171Z     Checking panic_unwind v0.0.0 (/checkout/src/libpanic_unwind)
2019-08-18T06:35:31.0762614Z     Checking hashbrown v0.5.0
2019-08-18T06:35:32.3382587Z error: unexpected close delimiter: `)`
2019-08-18T06:35:32.3383106Z    --> src/libstd/sys/wasi/fd.rs:232:44
2019-08-18T06:35:32.3383834Z     |
2019-08-18T06:35:32.3384261Z 232 |         wasi::sock_shutdown(self.fd, how) }).map_err(From::from)
2019-08-18T06:35:32.3384728Z 
2019-08-18T06:35:32.3397522Z error: incorrect close delimiter: `)`
2019-08-18T06:35:32.3397522Z error: incorrect close delimiter: `)`
2019-08-18T06:35:32.3398310Z    --> src/libstd/sys/wasi/fd.rs:141:10
2019-08-18T06:35:32.3398568Z     |
2019-08-18T06:35:32.3398841Z 132 |     ) -> io::Result<WasiFd> {
2019-08-18T06:35:32.3399151Z     |                             - un-closed delimiter
2019-08-18T06:35:32.3399810Z 141 |         ))?;
2019-08-18T06:35:32.3400112Z     |          ^ incorrect close delimiter
2019-08-18T06:35:32.3400172Z 
2019-08-18T06:35:32.3741484Z error: aborting due to 2 previous errors
2019-08-18T06:35:32.3741484Z error: aborting due to 2 previous errors
2019-08-18T06:35:32.3741820Z 
2019-08-18T06:35:32.3846445Z error: Could not compile `std`.
2019-08-18T06:35:32.3846537Z 
2019-08-18T06:35:32.3846788Z To learn more, run the command again with --verbose.
2019-08-18T06:35:32.3889284Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-08-18T06:35:32.3899193Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-08-18T06:35:32.3899273Z Build completed unsuccessfully in 0:02:51
2019-08-18T06:35:32.3953373Z == clock drift check ==
2019-08-18T06:35:32.3969710Z   local time: Sun Aug 18 06:35:32 UTC 2019
2019-08-18T06:35:32.3969710Z   local time: Sun Aug 18 06:35:32 UTC 2019
2019-08-18T06:35:32.4871313Z   network time: Sun, 18 Aug 2019 06:35:32 GMT
2019-08-18T06:35:32.4872147Z == end clock drift check ==
2019-08-18T06:35:38.7802484Z ##[error]Bash exited with code '1'.
2019-08-18T06:35:38.7850677Z ##[section]Starting: Checkout
2019-08-18T06:35:38.7853471Z ==============================================================================
2019-08-18T06:35:38.7853529Z Task         : Get sources
2019-08-18T06:35:38.7853594Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-18T06:41:41.0670806Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-18T06:41:41.0876630Z ##[command]git config gc.auto 0
2019-08-18T06:41:41.0965462Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-18T06:41:41.1029046Z ##[command]git config --get-all http.proxy
2019-08-18T06:41:41.1171820Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63676/merge:refs/remotes/pull/63676/merge
---
2019-08-18T06:42:16.3196721Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-18T06:42:16.3196964Z 
2019-08-18T06:42:16.3197418Z   git checkout -b <new-branch-name>
2019-08-18T06:42:16.3197655Z 
2019-08-18T06:42:16.3197871Z HEAD is now at 8326e8d07 Merge b8472cea1e8f40fe89ed673bd8bcd37a106c2e0d into fc8765d6d8623b2b5b4ca1d526ed1d7beb3fce18
2019-08-18T06:42:16.3352678Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-18T06:42:16.3355453Z ==============================================================================
2019-08-18T06:42:16.3355499Z Task         : Bash
2019-08-18T06:42:16.3355534Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-18T06:48:01.4101509Z 
2019-08-18T06:48:01.4101870Z error: incorrect close delimiter: `)`
2019-08-18T06:48:01.4102509Z   --> src/libstd/sys/wasi/time.rs:17:6
2019-08-18T06:48:01.4102841Z    |
2019-08-18T06:48:01.4104032Z 13 | fn current_time(clock: u32) -> Duration {
2019-08-18T06:48:01.4104595Z    |                                         - un-closed delimiter
2019-08-18T06:48:01.4105438Z 17 |     )).unwrap();
2019-08-18T06:48:01.4105907Z    |      ^ incorrect close delimiter
2019-08-18T06:48:01.4106064Z 
2019-08-18T06:48:01.4350086Z error: aborting due to 2 previous errors
2019-08-18T06:48:01.4350086Z error: aborting due to 2 previous errors
2019-08-18T06:48:01.4350207Z 
2019-08-18T06:48:02.4009850Z error: Could not compile `std`.
2019-08-18T06:48:02.4012067Z 
2019-08-18T06:48:02.4012724Z To learn more, run the command again with --verbose.
2019-08-18T06:48:02.4013897Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-08-18T06:48:02.4014372Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-08-18T06:48:02.4014531Z Build completed unsuccessfully in 0:02:47
2019-08-18T06:48:02.4014694Z == clock drift check ==
2019-08-18T06:48:02.4014854Z   local time: Sun Aug 18 06:48:01 UTC 2019
2019-08-18T06:48:02.4014854Z   local time: Sun Aug 18 06:48:01 UTC 2019
2019-08-18T06:48:02.4014997Z   network time: Sun, 18 Aug 2019 06:48:01 GMT
2019-08-18T06:48:02.4015138Z == end clock drift check ==
2019-08-18T06:48:08.0989245Z ##[error]Bash exited with code '1'.
2019-08-18T06:48:08.1042627Z ##[section]Starting: Checkout
2019-08-18T06:48:08.1044764Z ==============================================================================
2019-08-18T06:48:08.1044838Z Task         : Get sources
2019-08-18T06:48:08.1044887Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@newpavlov
Copy link
Contributor Author

@sunfishcode
Can you check correctness of sleep implementation?

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, the changes largely look good to me, except:

Ideally WASI target should be completely libc-free in my opinion.

This is not the purpose of the wasi target, that's the purpose of the wasm32-unknown-unknown target. The purpose of the wasi target is for Rust/C to interoperate well, so we shouldn't be removing integration with libc where it breaks that interoperation (like changes here to environment, abort, etc)

src/libstd/sys/wasi/mod.rs Outdated Show resolved Hide resolved
src/libstd/sys/wasi/os.rs Outdated Show resolved Hide resolved
src/libstd/sys/wasi/os.rs Show resolved Hide resolved
src/libstd/sys/wasi/os.rs Outdated Show resolved Hide resolved
@newpavlov
Copy link
Contributor Author

newpavlov commented Aug 19, 2019

This is not the purpose of the wasi target, that's the purpose of the wasm32-unknown-unknown target. The purpose of the wasi target is for Rust/C to interoperate well

WASI High-Level Goals does not state anything about it.

I believe it's quite important to be able to compile pure Rust applications for WASI target without bringing libc baggage, as it will allow us to minimize size of binary artifacts.

Using libc allocator is an understandable default, but if user will switch allocator to a pure-Rust one and will not use last_os_error anywhere in his application (and assuming manipulating env stuff will work via the Core API), then I think there is no reason to have libc in Rust std for WASI target.

Copy link
Member

@sunfishcode sunfishcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many of the changes here look good; I like using the safe and more-idiomatic interfaces in the wasi crate in general. But I'm concerned about the parts which impact C/Rust interop.

It's certainly useful to be able to support C and Rust interop. Quite a few popular Rust crates are *-sys wrappers around C/C++ libraries, for example.

Also, while you could theoretically re-implement crt1.c, libpreopen, etc., in Rust and eliminate libc entirely, this is code which will very likely be evolving as WASI evolves, so with my WASI maintenance hat on, I'm concerned that this would increase the overhead of making changes right now.

src/libstd/sys/wasi/os.rs Outdated Show resolved Hide resolved
src/libstd/sys/wasi/os.rs Outdated Show resolved Hide resolved
@newpavlov
Copy link
Contributor Author

I guess we will need a separate target for libc-free WASI then... :/ Something like wasm32-wasi-rust.

@newpavlov
Copy link
Contributor Author

I think I've reverted all potentially C-incompatible places. What do you think about adding a new libc-less WASI target in a separate PR?

@alexcrichton
Copy link
Member

This looks good to me, r=me with the wasi crate changes merged and published.

I do not think there is any practical interest currently in having a wasi target without the libc at https://github.com/cranestation/wasi-libc.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok this all looks great to me, thanks for pushing on this @newpavlov!

@sunfishcode would you be ok adding github:rust-lang:libs to the owner list of the wasi crate on crates.io? We currently tend to like to make sure there's not one point of failure for publishing to crates.io in case anything weird happens. AFAIK nothing weird has happened for years yet, we just want to make sure we're safe if need be!

src/libstd/Cargo.toml Outdated Show resolved Hide resolved
@alexcrichton
Copy link
Member

I'm also going to gently ping @rust-lang/libs on this for awareness. This is adding a new dependency to the standard library, the wasi crate, which is only used on the wasm32-wasi target. I've reviewed the crate myself and it looks good for inclusion, but if others have opinions on this now would be a good time!

@alexcrichton
Copy link
Member

Ok I'm going to go ahead and r+ this and we can sort out the logistics asynchronously

@bors: r+

@bors
Copy link
Contributor

bors commented Sep 5, 2019

📌 Commit 0662fcf has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 5, 2019
Centril added a commit to Centril/rust that referenced this pull request Sep 5, 2019
Upgrade rand to 0.7

Also upgrades `getrandom` to avoid bug encountered by rust-lang#61393 which bumps libc to `0.2.62`.
`wasi` crate was going to be added by rust-lang#63676 anyway.
Centril added a commit to Centril/rust that referenced this pull request Sep 5, 2019
bors added a commit that referenced this pull request Sep 5, 2019
Rollup of 5 pull requests

Successful merges:

 - #63676 (Use wasi crate for Core API)
 - #64094 (Improve searching in rustdoc and add tests)
 - #64111 (or-patterns: Uniformly use `PatKind::Or` in AST & Fix/Cleanup resolve)
 - #64156 (Assume non-git LLVM is fresh if the stamp file exists)
 - #64175 (Fix invalid span generation when it should be div)

Failed merges:

 - #63806 (Upgrade rand to 0.7)

r? @ghost
Mark-Simulacrum added a commit to Mark-Simulacrum/rust that referenced this pull request Sep 6, 2019
@bors
Copy link
Contributor

bors commented Sep 6, 2019

⌛ Testing commit 0662fcf with merge cc8c5a93376093421d83ead1e7ce12326a429272...

Centril added a commit to Centril/rust that referenced this pull request Sep 6, 2019
@Centril
Copy link
Contributor

Centril commented Sep 6, 2019

@bors retry rolled up.

bors added a commit that referenced this pull request Sep 6, 2019
Rollup of 10 pull requests

Successful merges:

 - #63676 (Use wasi crate for Core API)
 - #64094 (Improve searching in rustdoc and add tests)
 - #64111 (or-patterns: Uniformly use `PatKind::Or` in AST & Fix/Cleanup resolve)
 - #64156 (Assume non-git LLVM is fresh if the stamp file exists)
 - #64161 (Point at variant on pattern field count mismatch)
 - #64174 (Add missing code examples on Iterator trait)
 - #64175 (Fix invalid span generation when it should be div)
 - #64186 (std: Improve downstream codegen in `Command::env`)
 - #64190 (fill metadata in rustc_lexer's Cargo.toml)
 - #64198 (Add Fuchsia to actually_monotonic)

Failed merges:

r? @ghost
@bors bors merged commit 0662fcf into rust-lang:master Sep 6, 2019
@newpavlov newpavlov deleted the wasi branch September 6, 2019 11:41
bors added a commit to rust-lang/libc that referenced this pull request Sep 7, 2019
Remove WASI Core API

Closes #1434

This change does not break the backwards compatibility promise since WASI Core API is unstable right now. If applications or libraries want to use Core API directly they should use [`wasi`](https://crates.io/crates/wasi) instead of `libc`.

Blocked by: rust-lang/rust#63676

cc @sunfishcode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants